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OVERVIEW 

The  NATO  Modelling  and  Simulation  Group  (NMSG)  Symposium  ( MSG-045 )  “Transforming  Training 
and  Experimentation  through  Modelling  and  Simulation ”  was  conducted  in  Rome,  Italy  on  05  -  06 
October  2006.  All  sessions  of  the  Symposium  were  unclassified.  The  Symposium  audience  of 
approximately  100  persons  included  experts  from  NATO  countries,  Partners-for-Peace  (PfP)  nations,  as 
well  as  invited  nations. 

Out  of  39  abstracts  submitted,  20  Papers  were  selected  for  presentation.  In  addition,  two  invited  papers 
and  two  keynote  presentations  were  given.  This  technical  evaluation  report  summarizes  the  core  ideas  and 
results  presented  in  this  wide  variety  of  valuable  contributions  from  NATO  countries,  PfP  nations,  and 
invited  nations  by  capturing  the  main  ideas  in  the  flow  of  the  symposium  and  connecting  related  papers 
with  each  other.  Furthermore,  the  report  provides  an  overview  of  the  discussions  conducted  during  the 
symposium  following  each  presentation  and  gives  recommendations. 


1  INTRODUCTION 

The  importance  of  Modelling  and  Simulation  (M&S)  and  the  effect  it  has  had  on  transforming  training 
and  experimentation  in  NATO  and  national  activities  has  been  recognised  for  a  number  of  years  and  is 
reflected  in  several  symposium  proceedings.  Advances  in  low  cost,  high  power  computers,  graphics  and 
telecommunications/networking  are  some  of  the  key  technologies  that  have  provided  opportunities  for 
extensive  modelling  and  use  of  simulations  in  the  live,  virtual  and  constructive  domains  in  new  and 
exciting  ways.  Increasing  collaboration  with  related  NATO  panels  -  such  as  Information  System 
Technologies  (1ST)  and  Systems,  Analysis  and  Studies  (SAS)  -  and  other  related  organisations  -  such  as 
the  Simulation  Interoperability  Standards  Organization  (SISO)  and  the  Command  and  Control  Research 
Programme  (CCRP)  -  contributes  to  these  advancements.  As  a  result,  M&S  are  today  powerful  tools  that 
can  assist  in  many  ways  towards  the  Allied  Command  for  Transformation  (ACT)  mission  and  its 
commitment  to  the  transformation  of  NATO’s  military  capabilities.  This  symposium  focused  on  papers 
that  demonstrate  how  M&S  has  changed  component  command,  joint  training  and  doctrine  development. 
Additional  interest  is  the  influence  of  M&S  in  research  and  technology,  experimentation  into  Net  Enabled 
Capabilities  (NEC),  effect  based  operations  and  interoperability  for  the  effectiveness  of  NATO  and  the 
nations.  In  addition,  the  symposium  addressed  the  themes  of  research,  development  and  the 
application/adaptation  of  M&S  in  the  defence  and  commercial  environments.  The  Call  for  Papers 
explicitly  requested  papers  addressing  the  following  topics:  the  NATO  Response  Force,  Military 
Exercising,  Training  and  Decision  Support,  Network  Centric  Warfare,  and  Defence  against  Terrorism.  In 
addition  to  these  topics,  the  symposium  also  addressed  the  more  general  themes  of  research,  development 
and  M&S-related  standards.  Invited  Papers  dealt  with  the  NATO  Transformation  process  and  implication 
for  M&S,  the  NATO  Joint  Warfare  Centre  -  and  in  particular  their  current  and  future  applications  of 
M&S,  and  paper  from  Australia  regarding  their  use  of  M  &  S  and  the  development  of  Commercial  gaming 
technology  and  its  impact  in  military  applications. 
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2  KEYNOTE  PRESENTATIONS 

Two  keynote  presentations  by  high  ranking  officers  were  given  during  the  presentation. 

On  the  first  day,  RADM  Lucio  Accardo  from  the  Italian  Ministry  of  Defence  welcomed  the  guests  with  an 
overview  of  the  growing  importance  of  M&S  within  the  Italian  Armed  Forces.  He  focused  in  particular  on 
the  direct  connection  of  M&S  solutions  to  user  requirements.  Strong  connections  between  military  users 
and  M&S  engineers  are  therefore  essentials  in  all  application  areas.  At  the  same  time,  solid  education  is 
necessary  to  ensure  that  solutions  can  be  reused  and  emerging  technical  solutions  can  be  used  efficiently. 
Italy  is  now  using  M&S  support  in  all  operational  domains  and  technical  areas. 

On  the  second  day,  RADM  Jean-Louis  Kerignard,  from  NATO  HQ  Allied  Command  Transformation 
opened  the  session  with  a  keynote  emphasizing  the  role  of  M&S  as  a  key  enabler  for  reaching  the  goals  of 
ACT’s  Training  &  Experimentation  Transformation.  He  emphasized  the  growing  importance  of  M&S  for 
operational  use,  in  particular  the  role  of  supporting  operations.  The  role  of  MSG  recommendations  is 
essential  in  this  regard,  as  only  an  orchestrated  effort  of  the  nations  can  help  to  reach  this  goal.  Non 
aligned  individual  solutions  are  of  insufficient  use  in  NATO.  Technically  sound  solutions  enabling 
Networked  Simulation  &  Learning  Capabilities  Supporting  Education  and  Training  are  as  necessary  as  the 
tight  alignment  with  national  efforts,  among  which  the  US  Joint  Forces  Command  is  seen  as  the  primus 
inter  pares.  Common  challenges  are  to  find  enablers  for  a  Live,  Virtual  and  Constructive  Joint  synthetic 
capability,  common  scenarios,  and  better  reuse  of  national  contributions.  In  this  context,  he  emphasized 
the  importance  of  national  capabilities  in  supporting  NATO’s  efforts  particularly  on  the  Joint  Multi- 
Resolution  Model  (JMRM)1,  the  Networked  Simulation  &  Learning  Capabilities  Supporting  Education  & 
Training  (NETN),  and  the  Simulated  Mission  and  Rehearsal  Training  (SMART).  RADM  Kerignard 
explicitly  mentioned  the  contribution  of  MSG  Task  Groups  in  form  of  experimentation  and  demonstration 
and  emphasized  that  such  contributions  have  an  impact  on  ACT  Budget  and  Programme  of  Work. 


3  INVITED  PRESENTATIONS 

Two  invited  papers  were  presented  in  this  session. 

MP-MSG-045-01:  Dr  E  Cayirci  from  the  Joint  Warfare  Centre  (JWC)  in  Stavanger,  Norway,  gave  an 
overview  on  the  components  and  objectives  and  tasks  of  organisations  in  NATO  with  relevance  to 
Computer  Assisted  Exercises.  Training  remains  the  main  component  for  which  M&S  is  used,  but  other 
areas  are  of  growing  interest  (as  stated  by  RADM  Kerignard).  This  objective  is  supported  by  the  J3 
Organisations:  the  Joint  Warfare  Centre  (JWC)  in  Stavanger,  Norway,  the  Joint  Force  Training  Centre 
(JFTC)  in  Bydgoszcz,  Poland,  and  the  Joint  Analyses  and  Lessons  Learned  Centre  (JALLC)  in  Monsanto, 
Portugal.  These  centres  enable  NATO  to  supports  in  efforts  in  transformation,  training  and 
experimentation  using  modern  M&S  technologies  and  methods.  The  major  constraint  for  effective  training 
is  that  the  training  audience  is  distributed  and  uses  the  command  and  control  systems  they  use  in  real 
operations  as  well.  The  M&S  system  of  choice  is  currently  the  Joint  Theatre  Level  Simulation  (JTLS),  as 
it  has  been  for  several  years  now.  The  use  of  a  mature  solution  is  that  a  working  infrastructure  in  support 
of  JTLS  has  been  established  in  form  of  scenario  generation  solutions  for  JTLS,  the  JTLS  ICC  Mediation- 
ware  (JIM)  and  other  supporting  tools.  This  solution  is  currently  migrating  towards  the  Joint  Multi- 
Resolution  Model  (JMRM),  which  combines  the  high-resolution  model  JCATS  with  the  high-aggregated 
model  JTLS.  Currently,  scenario  modules  become  more  complex,  as  they  have  to  support  new  events  to  be 
injected  in  an  orchestrated  way  with  the  exercise  flow  and  the  training  objectives.  To  this  end,  events  and 
incidents  must  be  captured  and  managed,  which  is  done  in  an  event  and  incident  list.  The  current  tool  set 
still  focuses  on  JTLS,  but  support  of  JCATS  is  in  the  scope.  Figure  1  shows  the  target  infrastructure. 


1  The  first  invited  paper  (paper  MP-MSG-045-01)  presents  the  ideas  on  JMRM  and  its  use  in  NATO  in  more  detail. 
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Figure  1 :  JWC  JMRM  Target  Architecture 


It  should  be  pointed  out  that  JMRM  follows  the  ideas  (presented  by  MITRE  in  several  papers)  using  the 
entity  level  services  for  the  coupling,  which  means  that  the  information  exchange  happens  on  entity  level. 
Aggregation  or  disaggregation  is  task  of  the  federate,  currently  limited  to  JTLS  and  JCATS.  It  is  planned 
to  support  non-military  partners  as  well,  as  long  as  the  requirement  is  captured  for  the  exercise  to  be 
supported.  In  the  scope  of  this  technical  evaluation,  JMRM  for  JWC  can  become  a  milestone  on  the  way 
to  truly  international  federations.  Figure  1  can  easily  be  extended  into  multinational  federations  when 
allowing  other  federates  to  joint  the  architecture.  However,  the  use  of  open  standards  beyond  the  HLA  is 
necessary  to  fulfil  such  a  vision,  e.g.,  the  use  of  standard  FOMs  (such  as  the  Realtime-Platform-Reference 
FOM)  and  tools  that  support  multiple  federates.  The  current  standard  development  efforts  of  SISO,  in 
particular  the  Coalition  Battle  Management  Language  (C-BML,  see  paper  MP-MSG-045-13)  and  the 
Military  Scenario  Definition  Language  (MSDL),  are  candidates  to  enable  the  vision. 

MP-MSG-045-02:  In  the  second  invited  paper,  Mr.  Darren  McFarlane  gave  an  overview  of  recent  events 
and  developments  in  the  M&S  community  of  Australia.  Currently  Australia  cooperates  with  NATO  as  a 
‘Contact  Country'  and  enjoys  a  special  relationship  enabling  their  participation  and  collaboration  in  a 
number  of  technical  activities  and  is  deployed  as  an  international  partner  for  security  operations  in  many 
places.  The  Australian  Defence  and  Simulation  Office  (ADSO)  has  been  established  to  coordinate  and 
orchestrate  the  use  of  M&S  in  support  of  training  and  experimentation  supporting  these  activities.  Their 
work  is  captured  in  roadmaps.  The  Body  of  Knowledge  is  captured  in  a  Defence  Simulation  Manual 
(Volume  I  has  16  parts  and  deals  with  Defence  Simulation  Management;  Volume  II  comprises  five  parts 
on  focuses  on  Defence  Simulation  Application).  Studies  on  Return  of  Investments  were  conducted  in 
support  of  showing  the  necessity  of  using  M&S  to  transform  training  and  experimentation.  All  services 
support  M&S  activities  up  to  the  joint  level,  where  JTLS  and  JSAF  are  used  to  train  the  forces.  The 
Canberra  Based  Simulation  Centre,  the  Defence  Synthetic  Environment  initiative  and  the  Joint  Combined 
Training  Centre  working  tightly  connected  with  the  US  JNTC.  Australia  aims  to  broaden  their 
contributions  and  is  already  engaged  in  a  number  of  NATO  MSG  activities  and  tasks  in  particular  the 
exploitation  of  commercial  games  for  military  use.  Main  M&S  challenges  are  seen  in  simulating  cultural 
aspects,  resource  management,  the  coordination  of  forces,  obtaining  data  and  geography.  In  the  scope  of 
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the  technical  evaluation,  Australia  can  be  seen  as  an  example  of  non-traditional  partners  of  NATO,  which 
become  increasingly  important.  Australia  is  strongly  connected  with  the  US  JFCOM  -  as  also  envisioned 
by  RADM  Kerignard  in  his  keynote  -  Australia  also  has  a  chapter  of  SISO  to  deal  with  their  national 
needs  in  orchestration  with  SISO  recommendations  and  standardisation  efforts. 


4  TRANSFORMING  EXPERIMENTATION  -  NATO  &  INTERNATIONAL 
INITIATIVES 

Three  papers  were  grouped  into  this  track,  which  is  logically  accompanied  with  the  following  session 
focussing  on  the  national  counterparts  of  initiatives  presented  here. 

MP-MSG-045-03:  The  first  presentation  in  this  session  gave  an  overview  on  Rapid  Prototyping  during 
NATO  Experimentation  in  Support  of  the  CIMIC  Integrated  Capability  Team.  The  Crisis  Management 
Exercise  supported  by  this  team  is  currently  NATO’s  only  political  military  exercise,  although  additional 
exercises  with  similar  objectives  are  dealt  with  as  near- future  option  in  paper  MP-MSG-045-21.  Within 
this  team,  which  comprises  26  experimenters  from  NATO  and  the  nations,  M&S  is  mainly  used  to 
evaluate  innovations,  in  particular  to  be  used  to  do  some  sort  of  risk  assessment,  very  similar  to  the  use  as 
proposed  in  NATO’s  Code  of  Best  Practice  for  C2  Assessment.  Instead  of  a  using  the  traditional 
specification  driven  prototypes,  this  group  uses  prototype  driven  specifications.  The  power  of  this 
approach  became  obvious  when  the  options  for  the  Crisis  Management  Fusion  Centre  Model  and 
Simulation  were  set  up.  While  the  initial  idea  was  to  use  a  fully  orchestrated  and  harmonized  process 
view,  in  which  the  processes  within  the  Fusion  Cell,  Political  Cell,  Civil  Cell  and  Military  Cell  are  fully 
aligned  for  analyses,  the  simulation  driven  specification  led  to  the  definition  of  a  loosely  coupled  set  of 
cells  which  all  remained  independent  in  conducting  their  tasks.  Based  on  these  ideas,  the  cells  used  their 
own  simulation  based  on  executable  architectures  derived  from  DoDAF  compliant  structures.  This 
allowed  the  analysis  of  information  exchange  needs  as  well  as  actually  exchanged  information.  This 
structure  was  used  in  support  of  CIMIC  Experiment  Campaign  Plan,  which  is  one  of  ACT’s  first, 
coordinated,  coherent,  planned  experiment  series  utilising  M&S  and  which  focused  on  information 
collaboration  and  information  sharing.  M&S  was  not  only  used  in  support  of  the  exercise,  but  mainly  in 
analysis  of  the  necessary  information  exchange  between  CIMIC  partners,  i.e.,  between  NATO  and  non 
nationally  organized  groups.  It  should  be  pointed  out  that  the  selected  M&S  support  is  using  off-the-shelf 
solutions,  in  particular  Extend™.  This  new  approach  to  modelling  was  evaluated  not  only  to  be  effective, 
but  also  to  have  the  possibility  of  giving  a  greater  quality  and  variety  of  information  than  one  could  get 
from  the  large,  inflexible  models  that  one  sees  in  use  in  some  other  places.  In  the  scope  of  the  technical 
evaluation  the  usability  of  alternative  simulation  tools  is  the  most  important  aspect  of  this  paper.  It  should 
be  read  together  with  paper  MP-MSG-045-21,  in  which  the  NATO-developed  tool  Gamma  is  used  in  a 
similar  context.  Both  solutions  have  their  value  and  should  be  seen  complimentary,  not  excluding. 

MP-MSG-045-04:  The  second  presentation  of  this  session  focused  on  lessons  learned  from  the 
Multinational  Experiment  (MNE  4)  as  seen  from  the  US  Joint  Forces  Command  (USJFCOM)  Joint 
Innovation  &  Experimentation  Directorate  (J9).  The  paper  presents  the  technical  approach  used  to  create  a 
sufficient  M&S  environment  for  MNE  4,  how  this  development  effort  may  be  leveraged  for  the  upcoming 
experiment  MNE  5  and  how  this  work  can  support  the  development  and  integration  of  joint,  interagency 
and  multinational  capabilities  to  meet  present  and  future  operational  needs.  The  group  decided  to  use  a 
hierarchical  approach  with  well  defined  tasks  for  the  participating  systems  in  support  of  effect  based 
operations  targeting  all  elements  of  the  Political  Military  Economic  Societal  Information  Infrastructure 
(PMESII)  domain.  Simulations  are  used  to  show  strength  and  weaknesses  of  partners,  allies  and 
opponents.  The  simulation  systems  Alliance  (FRA),  Joana  (DEU)  and  Joint  Semi-automated  Forces 
(JSAF,  USA)  were  coupled  using  HLA  to  produce  a  consistent  simulated  operation.  Every  simulation 
contributed  in  the  area  it  was  best  to  use  for,  and  all  nations  could  use  their  simulation  system  to  evaluate 
their  ideas  in  the  international  context.  This  is  very  close  to  the  NMSG  Pathfinder  vision.  Additionally  to 
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setting  up  the  federation,  C2  interfaces  to  produce  the  common  operational  pictures  (COPs)  needed  by  the 
training  audience  needed  to  be  set  up.  Figure  2  shows  the  hierarchy  of  COPs. 


Of  particular  interest  is  the  use  of  Information  Exchange  elements  based  on  the  Joint  Consultation 
Command  and  Control  Information  Exchange  Data  Model  (JC3IEDM),  which  follows  a  recommendation 
given  in  the  proceedings  of  the  NATO  MSG  Symposium  2003  on  C3I  and  M&S  systems  interoperability. 
MNE4  will  be  used  as  a  logical  starting  point  for  PMESII  and  will  be  extended.  It  should  be  noted  that 
JIM  was  used  for  injects.  It  is  planned  to  use  the  newly  initiated  PMESII  Centre  of  Excellence  to  continue 
the  work.  Paper  MP-MSG-045-21  will  give  a  slightly  different  view  on  MNE  4  from  the  perspective  of 
NC3A  participation.  It  should  also  be  mentioned  that  V&V  was  not  done  beyond  the  face- value  testing. 

MP-MSG-045-05:  The  third  presentation  on  “ Modelling  &  Simulation  for  Experimentation ,  Test  & 
Evaluation  and  Training:  Alenia  Aeronautica  experiences  and  perspectives ,”  must  be  regarded  as  an 
example  how  M&S  is  used  to  transform  procurement  and  training  processes  and  how  reusable 
components  are  identified  and  applied.  The  presenting  company  is  a  main  contributor  to  the  Eurofighter 
projects.  In  order  to  meet  the  rapidly  changing  requirements,  individual  components  are  no  longer 
sufficient.  Using  the  example  of  a  common  synthetic  environment  for  all  simulators,  several  domains  are 
identified,  such  as  Human-Machine  Interfaces,  Image  Generation  and  Geographical  Databases,  Tactical 
Scenarios,  and  more.  Applying  common  components  facilitates  to  multi-purpose  use  of  this  netted 
components,  such  as  the  use  for  test  flight  rehearsal  and  optimization  with  simulators  that  were  designed 
for  training.  Another  interesting  aspect  of  the  paper  are  the  lessons  learned  from  the  Joint  MSG-001/SAS- 
034  “Exercise  First  Wave”,  which  was  a  multinational  exercise  of  real-time  distributed  simulation, 
involving  seven  participating  NATO  Nations  with  several  flight  simulators.  The  presentation  gave 
examples  for  the  identified  domains  using  products  of  the  presenting  company.  Of  high  importance  for 
NATO  is  the  identification  of  underlying  open  standards  for  such  components  to  insure  the  information 
exchange  and  also  the  use  of  potential  alternatives  for  such  components  (such  as  using  an  alternative 
image  generator  in  exchange  for  the  proposed  solution,  if  this  satisfies  the  NATO  requirements  better). 

This  presentation  can  be  seen  as  a  connection  between  the  international  and  the  national  efforts  described 
in  the  following  section.  The  use  of  national  framework  recommendations  for  international  collaboration 
is  also  topic  of  other  paper  in  these  proceedings,  such  as  papers  9  and  20. 
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5  TRANSFORMING  EXPERIMENTATION  -  NATIONAL  EFFORTS 

As  the  previous  session,  this  session  also  comprised  three  papers,  but  it  emphasizes  national  efforts  on 
Transforming  Experimentation.  It  completes  the  session  on  internationally  efforts. 

MP-MSG-045-06:  The  session  on  national  efforts  started  with  the  UK  contribution  on  “The  Use  of  a 
Conceptual  Battlespace  Architecture  (CBA)  to  Manage  and  Exploit  Concepts  and  Doctrine 
Experimentation”.  One  aspect  of  particular  interest  of  their  approach  is  to  define  an  architecture 
(MODAF)  driven  approach  for  reuse  of  contributions.  One  of  the  central  questions  for  such  a  concept  is: 
how  to  measure  success?  As  the  objective  is  to  use  M&S  support  for  experimentation  as  a  means  to  enable 
effective  definition,  development  and  delivery  of  NEC  &  Effects  Based  Operations  (EBO),  this  question 
can  also  be  formulated  as:  How  can  the  findings  from  a  range  of  experiments  be  combined  to  add  value? 
The  recommended  solution  is  very  close  to  the  recommendations  summarized  in  the  NATO  Code  of  Best 
Practice  for  C2  Assessment,  namely  to  use  a  layered  conceptual  battlespace  architecture  which  is  depicted 
in  detail  in  Figure  3. 
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Figure  3:  An  overview  of  the  elements  of  the  conceptual  battlespace  architecture 

The  resulting  framework  allows  the  recognition  of  patterns  and  the  reuse  of  such  patterns  for  similar 
operational  nodes  (such  as  the  general  decision  making  process  can  be  adapted  for  different  instantiations 
of  headquarters  after  level  and  type- tailored  specifications).  However,  this  derivation  of  knowledge  is 
done  by  an  analyst  and  with  current  technology  automation  is  limited.  Creating  general  knowledge  from 
specific  experiments  is  even  more  challenging.  In  summary,  CBA  is  an  approach  to  elicit  additional  value 
from  a  range  of  experimentation,  doctrine,  concepts  and  architectures.  Direct  impact  on  current  work  in 
the  MSG  is  seen  for  the  Task  Groups  MSG-027  and  MSG-052.  The  work  of  C-BML  -  in  particular  the 
current  definition  of  a  set  of  controlled  vocabularies  -  is  of  interest  as  well. 

MP-MSG-045-07 :  The  following  Canadian  contribution  to  this  session  focuses  on  capability  engineering 
on  the  system  of  systems  level.  Core  idea  is  to  use  the  Department  of  Defense  Architecture  Framework 
(DoDAF)  artefacts  to  define  systems  capabilities  as  the  first  step  and  then  use  executable  architectures  to 
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represent  system  in  the  second  step.  This  allows  using  capability  engineering  methods  to  identify  gaps  in 
static  (using  the  artefacts)  and  dynamic  (executing  the  architecture)  modes.  The  standard  use  of  DoDAF  is 
to  use  the  operational  view  to  say  is  needed  and  then  using  the  system  view  to  say  what  is  delivered.  The 
use  of  DoDAF  is  slightly  different:  While  the  operational  view  still  defines  the  required  capabilities,  the 
system  view  shows  which  capabilities  are  delivered  by  each  system  of  the  system  of  systems.  A  traffic- 
light-code  is  used  to  add  information  to  this  matrix.  If  the  applicable  systems  are  using  the  same 
engineering  approach,  solutions  become  now  exchangeable.  Furthermore,  the  architecture  products 
support  the  integration  of  underlying  M&S  systems.  These  ideas  were  successfully  applied  in  a  six  week 
experiment.  In  the  scope  of  the  technical  evaluation  it  is  recommended  to  have  a  closer  look  at  the 
applicability  of  complementing  standards,  such  as  UML  and  XML,  to  capture  artefacts  DoDAF  artefacts, 
such  as  described  in  more  detail  in  the  2006  special  issue  of  the  Journal  for  Defense  Modeling  and 
Simulation  (JDMS;  Applications,  Methodology,  Technology)  on  “Integrated  System  Evaluation:  a 
Consumer/Warfighter  Perspective.”  The  cascading  use  of  Architecture  Documentation  based  on  DoDAF 
(or  UK  Ministry  of  Defence  Architectural  Framework  (MoDAF),  as  done  in  paper  MP-MSG-045-06) 
followed  by  Process  Modelling  and  the  execution  using  executable  architectures  followed  by  Virtual  and 
Constructive  Simulation  is  an  applicable  scheme  to  analyse  and  evaluate  heterogeneous  systems  and  their 
operational  support.  In  addition  to  the  paper,  however,  it  can  be  recommended  to  add  system  dynamics  as 
a  means  before  using  architectures.  Furthermore,  the  use  of  agents  to  evaluate  “hot  spots”  or  bottlenecks 
in  the  necessary  detail  is  recommended.  Some  similar  ideas  are  also  mentioned  in  papers  4,  1 1  and  21. 

MP-MSG-045-08:  The  Spanish  contributions  highlighted  the  use  of  applications  of  operational  research 
methods  and  the  use  of  simulation  as  one  embedded  tool  in  the  suite  of  mathematical  programming, 
heuristics  and  statistics.  The  paper  shows  furthermore  the  integratability  of  supporting  related  ideas,  such 
as  ontologies,  the  different  implementation  models  of  the  model  driven  architecture  (MDA)  approaches  as 
recommended  by  the  Object  Management  Group  (OMG)  and  other  ideas,  such  as  MSG-042  inputs, 
findings  and  recommendations.  The  ideas  are  used  in  support  of  the  Spanish  Navy  Operations  Research 
Laboratory  (GIMO).  The  objective  of  their  work  is  to  improve  the  reusability  of  components.  One  of  the 
identified  requirements  is  to  avoid  semantic  ambiguity  of  resources  and  is  supported  by  an  agreed  formal 
semantic  definition  of  resources  Repository  of  tools  can  than  rely  on  the  unambiguous  repository  of  terms. 
To  support  these  unambiguous  definitions  on  resource  (component)  and  term  (repository)  level,  ontology 
research  is  applied.  In  the  opinion  of  the  author  of  this  technical  evaluation,  the  main  challenge  will  be  the 
underlying  assumption  that  a  common  ontology  can  be  defined  and  will  be  used  in  NATO.  Based  on  the 
standards  literature  on  the  ontological  spectrum,  a  machine-understandable  ontology  as  recommended  in 
this  paper  can  be  defined  as  a  data  model  enriched  by  axiomised  business  rules.  Therefore,  a  rigorous  data 
and  metadata  management  is  the  necessary  requirement  to  make  such  an  effort  successful.  It  is  very 
unlikely  that  all  NATO  nations  and  partners  will  exclusively  agree  on  one  data  definitions.  Instead  of  a 
common  ontology,  the  use  of  a  meta-ontology  enabling  the  capturing  and  mapping  of  national  models  is 
an  option.  Such  an  idea,  based  on  the  ideas  of  federated  heterogeneous  databases,  is  described  in  paper 
MP-MSG-045- 1 6. 

6  INTEROPERABILITY  ISSUES 

Many  authors  responded  to  the  call  for  papers  with  interoperability  focused  contributions,  so  that  two 
sessions  were  instantiated  for  this  workshop. 

MP-MSG-045-09:  The  first  paper  presented  in  this  session  gave  an  update  on  joint  efforts  to  deal  with  the 
challenges  of  live,  virtual  and  constructive  simulation  coupling.  The  US  Joint  Forces  Command,  the  Joint 
National  Training  Capability  (JNTC)  copes  with  this  related  task.  JNTC  was  already  mentioned  in  the 
keynote  of  RADM  Kerignard  as  a  strong  partner  for  ACT  as  well  as  during  the  second  invited 
presentation,  as  Australia’s  joint  combined  training  centre  is  connected  with  it.  JNTC  can  serve  as  a  model 
for  NATO  as  the  challenges  dealt  with  are  similar  to  those  being  observed  by  NATO,  currently  multiple 
protocols,  gateways,  and  object  models  are  often  used  to  bring  an  LVC  Environment  together.  The  vision 
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is  to  gradually  merge  the  different  architectures  often  used,  such  as  HLA,  TENA  and  DIS  -  into  one 
integrated  LVC  infrastructure.  As  a  first  step  in  this  direction,  the  Joint  Live  Virtual  Constructive  Data 
Translator  (JLVCDT)  is  currently  developed  to  support  a  selected  set  of  first  architecture  candidates  as  a 
sort  of  common  gateway  on  the  way  to  a  common  architecture.  JLVCDT  was  designed  to  improve  multi¬ 
service  LVC  component  interoperability.  It  is  envisaged  that  JLVCDT  will  deliver  an  open  architecture 
via  a  public  API  to  encourage  broad  adoption  in  existing  LVC  environments.  JLVCDT  and  its  plug-in 
modules  will  be  JLCOM-certified  in  the  Joint  Advanced  Training  Technology  Laboratory  (JATTL). 
JLVCDT  has  been  used  in  different  international  environments,  including  but  not  limited  to  MSG-027 
experiments.  It  is  planned  to  make  the  tool  available  in  controlled  contexts,  i.e.  the  open  architecture 
which  is  government  owned.  A  timeline  for  release  of  the  tool  is  not  yet  established. 

MP-MSG-045-10:  The  second  paper  on  “Verification,  Validation  and  Accreditation  (VV&A)  Leveraging 
International  Initiatives”  not  only  gives  an  overview  of  the  importance  of  VV&A,  it  also  highlights  the 
possibility  of  leveraging  knowledge  and  results  already  available  through  other  organisations.  The  paper 
describes  high-level  views  of  the  following  initiatives:  NATO  Modelling  and  Simulation  Group  Task 
Group  NMSG-19;  Simulation  Interoperability  Standards  Organization  (SISO):  Verification,  Validation  & 
Accreditation  Overlay  to  Lederation  Development  Product  Development  Group  (PDG)  and  SISO  Generic 
Methodology  for  VV&A  Study  Group,  and  the  European  initiative  Referential  for  VV&A  (REVVA). 
Details  on  the  individual  initiatives  are  given  in  the  paper.  In  the  context  of  this  technical  evaluation  it 
should  be  noted  that  this  paper  focuses  much  more  on  education  and  procedural  support  than  the  other 
more  technology  based  papers  in  this  session.  It  clearly  supports  the  view  that  conceptual  problems  cannot 
be  solved  with  tools  on  the  technical  level,  but  that  a  combined  effort  of  efficient  tool  support  paired  with 
sufficient  education  in  common  engineering  processes  is  necessary.  It  should  be  further  noted  that 
Accreditation  is  the  formal  process  to  be  applied  on  the  national  level,  the  paper  used  Acceptance  instead 
of  Accreditation ,  which  is  the  generic  process  that  can  lead  to  national  accreditation  established  in  the 
context  of  the  user.  It  is  also  an  interesting  observation  that  VV&A  has  not  been  conducted  to  the  full 
recommended  extend  in  any  of  the  presented  application  cases  during  this  symposium,  although  it  has 
been  identified  as  a  future  activity  in  papers  4,  20  and  21. 

MP-MSG-045-11:  The  third  paper  deals  with  “Effectiveness  of  JSAF  as  An  Open  Architecture,  Open 
Source  Synthetic  Environment  in  Defence  Experimentation”,  in  which  the  authors  critically  reflect  on 
practical  reuse  issues  to  evaluate  new  mission  roles  and  to  develop  tactics  for  new  missions  and  new  types 
of  equipment.  The  driving  concept  behind  this  evaluation  is  the  idea  to  allow  scalable  fidelity,  i.e.,  the 
ability  to  support  current  and  future  requirements  ranging  from  high  to  low  level  of  fidelity.  The 
presentation  did  not  exclusively  focus  on  JSAF,  but  looked  at  other  efforts  as  well.  However,  what  is  of 
significant  interest  in  the  context  of  this  technical  evaluation  is  the  rigorous  application  of  open 
architectures  and  open  international  standards.  Lessons  learned  and  recommended:  pay  a  bit  more  and 
have  more  flexibility,  but  you  need  a  sets  of  metrics  to  back  your  recommendations.  Open  source  or 
Government  Source  Available  solutions  have  been  proven  to  be  supportive  of  the  ideas  presented  in  the 
paper.  It  may  be  noted  that  during  the  recent  Simulation  Interoperability  Workshop  Fall  2006  by  SISO  the 
topic  “Open  Source  and  its  Contribution  to  Simulation  Interoperability”  was  the  theme  of  the  Academic 
Night.  The  insights  of  the  participating  experts  from  various  universities  may  be  a  good  additional  source 
for  the  interested  reader. 

MP-MSG-045-12:  The  fourth  paper  presented  ongoing  work  of  research  conducted  for  the  Federal  Armed 
Forces  of  Germany  on  scalable  solutions  for  the  new  Information  Technology  in  support  of  simulations 
for  the  Bundeswehr  (ITSimBw).  The  proposed  infrastructure  is  scalable  concerning  functionality  (how 
detailed  are  functions/effects  modelled/simulated?),  time  (in  which  time  steps  will  the  simulation 
proceed?)  and  space  (which  spatial  scale  will  be  chosen  for  modelling?).  A  second  important  aspect  is  the 
usability  of  intelligent  software  agents  in  such  an  environment.  The  third  new  idea  presented  in  this  paper 
is  the  use  of  an  exact  3D  representation  of  all  physical  entities,  as  this  is  considered  to  be  critically 
important  for  a  realistic  representation  of  the  environment  within  a  military  simulation.  The 
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recommendation  is  to  use  voxels  (pixel-like  representations  of  volumetries,  regular  volumes  of  space). 
The  coupling  with  legacy  systems  is  supported  by  wrappers.  The  project  is  still  in  the  experimentation 
phase  but  shows  some  promising,  but  tentative  first  results.  Furthermore  it  should  be  pointed  out  that  the 
work  presented  is  not  a  framework  for  mediated  integration  of  existent  solutions,  such  as  proposed  in 
earlier  papers,  but  the  implementation  of  a  new  model  paradigm.  It  is  the  opinion  of  the  author  of  this 
technical  evaluation  that  more  research  and  experimentation  is  necessary  before  this  approach  reaches  the 
maturity  of  solutions  as  used  in  JWC  or  the  JFCOM.  The  underlying  vision,  however,  is  aligned  with 
several  of  the  Pathfinder  ideas  and  activities  and  it  is  recommended  to  observe  technical  developments  and 
methodological  contributions  in  these  areas  and  continue  to  evaluate  their  maturity  level  for  the  practical 
use  within  NATO. 

MP-MSG-045-13:  The  fifth  paper  highlighted  the  recent  developments  in  Coalition  Battle  Management 
Language  (C-BML).  The  SISO  Study  Group  recommended  to  define  C-BML  as  the  unambiguous 
language  used  to  command  and  control  forces  and  equipment  conducting  military  operations  and  to 
provide  for  situational  awareness  and  a  shared,  common  operational  picture.  Several  BML  tasks  are 
currently  under  development  by  national  as  well  as  international  expert  groups  presented  in  the  paper.  A 
common  denominator  is  the  use  of  a  common  data  model  to  capture  the  information.  C-BML  uses  the 
Command  and  Control  Information  Exchange  Data  Model  (C2IEDM),  the  migration  to  next  generation  of 
this  data  model,  the  Joint  Consultation  Command  and  Control  Information  Exchange  Data  Model 
(JC3IEDM),  is  currently  being  prepared.  This  use  of  this  data  model  follows  the  recommendation  given  in 
the  summary  of  the  2003  NATO  MSG  Symposium,  which  focused  on  C3I  and  M&S  Interoperability. 
Also,  the  BML  idea  was  presented  to  the  MSG  community  during  the  2004  NATO  MSG  Symposium  in 
paper  MP-MSG-028-12.  Of  particular  interest  in  the  scope  of  this  technical  evaluation  is  the  report  on  the 
MSG  Exploratory  Team  on  C-BML  (ET-016),  the  current  state  of  MSG’s  Task  Group  on  C-BML  (MSG- 
048),  and  the  connections  to  the  Simulation  Interoperability  Standards  Organization  (SISO)  Product 
Development  Group  (PDG)  on  C-BML.  The  work  is  ongoing,  relevant  projects  are  referenced  in  detail  in 
the  paper.  In  the  discussion  following  the  presentation  it  was  noted  that  -  although  the  paper  uses  the 
complete  recommended  definition  -  the  presentation  exclusively  deals  with  the  reporting  aspect  of  C- 
BML.  The  reason  for  this  is  that  the  first  phase  of  the  standardization  efforts  of  SISO  also  focuses  on  this 
aspect  and  MSG  decided  to  use  the  SISO  recommendation  to  avoid  the  introduction  of  unnecessary  and 
avoidable  ambiguities. 

MP-MSG-045-14:  The  sixth  paper  summarizes  the  lessons  learned  in  the  form  of  a  success  story  on  an 
effort  conducted  by  MBDA  Missile  Systems.  MBDA  is  present  in  France,  Italy  and  the  UK.  They 
reported  on  current  work  targeting  a  European  Distributed  Experimentation  Capability  in  support  of  their 
distributed  and  international  company  that  grew  together  from  historically  independent  technical  centres. 
Their  recommendations  are  insofar  of  interest  to  the  MSG,  as  they  aim  for  a  common  technical  approach, 
based  on  a  common  architecture,  a  common  infrastructure  and  common  components  to  migrate  existing 
and  applicable  solutions  into  a  common  framework.  The  Synthetic  Environment  Centre  of  Excellence  of 
MBDA  coordinated  the  efforts,  using  web  based  tools  for  workshops,  user  guides,  video  conference  and  a 
web  based  repository.  This  approach  allowed  the  reuse  of  applications  and  components  distributed  and 
developed  separately  at  the  various  sites  of  the  company  online  in  a  collaborative  environment.  The  paper 
presents  the  currently  supported  distributed  architecture  between  MBDA  in  France,  Italy  and  UK.  It  also 
reports  on  cost  savings  and  the  consolidating  research  activities  based  on  sharing  best  practice  and  shared 
tools.  To  what  degree  these  results  can  be  used  for  NATO  remains  to  be  evaluated  in  detail,  as  the 
presented  effort  was  international,  but  conducted  with  the  borders  of  MBDA  Missile  Systems.  However, 
in  the  light  of  the  vision  to  establish  a  Networked  Simulation  &  Learning  Capabilities  Supporting  the 
NATO  Education  &  Training  Network  (NETN),  in  particular  the  technical  solutions  for  web-based 
distributed  collaboration  are  candidates  for  more  intensive  evaluation. 

MP-MSG-045-15:  The  final  paper  of  the  session  on  interoperability  issues  deals  with  Interfacing 
Simulations  with  Training  Content.  It  is  another  collaboration  of  Aviation  Industry  CBT  Committee,  IEEE 
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Learning  Technology  Standards  Committee  (LTSC)  and  SISO  of  immediate  interest  to  MSG.  The  paper  is 
primarily  based  on  SISO  Study  Group  results  and  recommendation,  which  comprises  many  LTSC 
recommendations.  This  group  was  formed  to  study  possible  standardization  for  the  integration  of 
simulation-based  learning  experiences  with  Sharable  Content  Object  Reference  Model  (SCORM) 
environments.  The  paper  and  presentation  comprises  many  technical  details  of  interest.  However,  in  scope 
of  the  technical  evaluation  the  recommendation  to  focus  on  standards  for  information  exchange  (i.e. 
notional  data  models)  and  architectural  constraints  is  of  particular  interest.  Beside  the  details  of  the  study 
group  the  paper  also  comprises  a  description  of  the  working  procedures  of  SISO.  SISO  has  now  been 
accepted  as  a  standardization  body  by  NATO,  this  means  that  SISO  standards  can  become  NATO 
Standardization  Agreements  and  as  the  study  groups  and  product  development  groups  are  open  to  the 
public  this  means  that  NATO  MSG  Exploratory  Teams  and  Task  Groups  can  use  SISO  to  recommend 
standardization  work.  It  is  valuable  to  know  and  understand  these  constraints.  The  paper  is  accompanied 
by  paper  MP-MSG-045-17  in  these  proceedings. 


7  TRANSFORMING  TRAINING,  EDUCATION  &  DECISION  SUPPORT 

As  the  track  on  Interoperability  Issues,  the  track  on  Transforming  Training,  Education  and  Decision 
Support  was  targeted  by  many  authors  and  filled  several  sessions. 

MP-MSG-045-16:  The  session  was  opened  with  an  overview  brief  on  Pathfinder  activities  and 
recommendations  for  the  way  forward  given  in  paper  MP-MSG-045-16  on  a  “Web  Portal  Approach  for 
Knowledge  Networks  in  Support  of  the  Pathfinder  Programme”.  Although  rooted  in  MSG-027,  this  paper 
is  not  a  presentation  of  the  task  group  findings,  but  it  is  intended  to  place  the  MSG-027  into  the  bigger 
picture  of  other  task  groups,  the  advanced  distance  learning  ideas,  and  the  new  ideas  for  the  NATO 
Education  &  Training  Network  (NETN).  The  paper  summarizes  the  important  aspects  of  NATO’s  M&S 
Master  Plan  and  the  resulting  flagship  of  the  MSG:  Pathfinder.  This  effort  can  be  considered  as  being  the 
yardstick  for  all  technical  activities  and  must  therefore  itself  follow  a  set  of  guiding  principles  and  rules. 
The  presentation  recommended  that  future  MSG  activities  should  be  measured  to  what  degree  they 
support  activities  like  Pathfinder  and  NETN  and  should  not  exclusively  produce  task  force  specific  results. 
The  presentation  focused  on  two  aspects  given  in  the  vision  of  Pathfinder.  Firstly,  instead  of  using  one 
mandated  or  selected  system,  every  nation  brings  their  national  systems.  These  systems  can  be  federated 
in  a  timely  way  into  the  common  solution.  The  idea  behind  this  is  that  national  ideas  are  nowhere  better 
modelled  than  in  the  national  simulation  systems.  JMRM  described  earlier  in  paper  MP-MSG-045-01  is 
not  such  a  solution,  but  is  leading  in  this  direction  as  it  in  principle,  allows  Users  to  federate  with  other 
systems  than  JTLS  and  JCATS.  Secondly,  M&S  is  only  relevant  when  being  applied  to  the  benefit  of  the 
NATO  user.  To  enable  this,  the  convergence  of  operational  systems  and  M&S  is  mandatory.  The  use  of 
common  standards  and  data  models  as  recommended  and  demonstrated  in  several  other  papers  of  this 
symposia  and  all  point  to  the  same  direction.  The  presentation  focused  on  the  web  portal  that  was 
designed  to  support  knowledge  for  the  Pathfinder  Integration  Environment  (MSG-027)  and  how  to  reuse 
the  portal  and  the  ideas  to  bring  Pathfinder  forward.  The  Pathfinder  Integration  Environment  (PIE) 
Knowledge  Web  Portal  is  accessible  from  NATO  ACT  Web  Portal  http  :  //pathfinder  .  vmasc  . 
odu.edu/pathfinder/  and  describes  M&S  Resources  in  form  of  a  repository  as  well  as  M&S 
Integration  Knowledge  in  form  of  use  case,  processes  and  application  steps  using  the  resources  described 
in  the  repository.  The  use  of  meta-schemas  gives  the  user  maximal  flexibility  and  allows  the  import  of 
other  repository  information  based  on  their  XML  schema  and  support  of  W3C  compliant  web  services. 
The  presentation  stressed  the  necessity  to  apply  such  open  standards  and  architectures  specified  to  the 
level  of  detail  needed  to  align  and  orchestrate  the  contributing  technical  activities  and  task  group  results. 

MP-MSG-045-17:  The  second  paper,  that  accompanies  paper  MP-MSG-045-15  of  these  proceedings, 
deals  with  “Instructional  Design  Considerations  and  Planning  to  Transform  Simulation  Systems  to  a 
Platform  for  Delivery  of  Instruction”.  The  paper  focuses  on  the  necessity  to  better  deal  with  Simulation 
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Based  Instruction  (SBI).  Current  paradigms,  such  as  following  a  strict  Analysis,  Design,  Development, 
Implementation  and  Evaluation  (ADDIE)  process,  are  not  appropriate  and  improvements  are 
recommended.  However,  emerging  standards  exist  in  support  of  the  community,  such  as  Advanced 
Distributed  Learning  (ADL),  Sharable  Content  Object  Reference  Model  (SCORM),  Content  Object 
Repository  Discovery,  Registration  Architecture  (CORDRA),  Sharable  Content  Object  (SCO)  and 
Learning  Management  System  (LMS).  These  standards  do  not  solve  the  problem,  but  they  contribute  to 
the  solution.  Paper  and  presentations  show  the  need  for  formal  instructional  design  method  for  SBI.  This 
instructional  design  for  simulation  should  be  an  independent  discipline  from  the  simulation  design  process 
and  should  be  independent  of  development  process,  independent  of  underlying  models  and  independent  of 
specific  implementation.  While  simulation  models  are  based  on  engineering  principles2,  instruction  and 
student  models  must  be  based  on  learning  principles.  Also,  instead  of  seeing  the  process  as  composed  of  a 
series  of  stages,  a  spiral  development  using  the  components  Ontology,  Process,  Collaboration  model  and 
Product  representation  is  proposed.  In  order  to  increase  its  effectiveness,  SBI  must  be  improved.  The  pure 
federation  of  simulation  systems  is  not  sufficient  in  support  of  these  ideas.  However,  the  need  for 
metadata  in  support  of  not  only  composable  engineering  principles,  but  also  pedagogically  composable 
modules  and  the  possibility  to  support  multiple  levels  of  abstraction  is  made  in  this  paper.  It  is  worth 
mentioning  that  these  recommendations  don’t  minimize  the  aspect  of  technical  achievements,  such  as 
federation  development  on  a  big  scale.  It  adds  another  dimension  to  the  problem  scale  that  so  far  was  not 
sufficiently3  regarded. 

MP-MSG-045-18:  The  third  paper  summarizes  Norwegian  efforts  on  setting  up  an  infrastructure  in 
support  of  Distributed  Simulation  Based  Joint  Tactical  Training.  The  experiment  JADE  was  conducted 
with  the  goal  to  demonstrate  and  explore  distributed  simulation  based  training  of  procedures  and  air 
picture  production  between  naval  units  and  a  Control  and  Reporting  Centre  (CRC).  The  hypothesis  for  this 
national  experiment  was  that  legacy  stand  alone  training  simulators  can  be  federated  as  a  first  step  towards 
a  joint  training  capability.  This  is  increasingly  important  as  Network  Based  Defence  (NBD)  is  likely  to 
increase  the  need  for  collective  and  joint  training.  The  synthetic  environment  for  joint  and  combined  air 
defence  covered  training  simulators,  a  C2  system,  tactical  data  link  network,  voice  communications  and 
simulation  data  exchange.  The  resulting  infrastructure  dealt  with  live  entities  and  simulated  entities  and  is 
depicted  in  Figure  4.  While  HLA  1.3  was  the  technical  backbone,  DIS-adaptors  were  used  to  integrate 
these  simulators.  Voice-over-IP  and  Tactical  Data  Links  were  supported.  One  of  the  resulting  challenging 
aspects  of  integrating  operational  C2  systems  was  security.  The  results  of  the  evaluations  were  positive. 
With  regard  to  a  technical  evaluation  it  should  be  noted  that  the  technical  achievements  are  remarkable. 
However,  the  benefit  of  such  experiments  and  other  national  efforts  discussed  earlier  or  later  are  no 
exception,  could  have  been  significantly  improved  by  a  more  rigorous  application  of  open  standards  (HLA 
IEEE  1516  instead  of  HLA  1.3)  and  integration  results  and  lessons  learned  for  such  important  questions  as 
security  solutions,  integration  of  standardized  C3I  system  interfaces  (such  as  tactical  data  links),  etc  would 
have  been  captured  in  a  form  that  it  could  be  shared  with  NATO  nations  and  partners.  Paper  MP-MSG- 
045-20  presents  another  very  similar  approach. 


2 

The  Federation  Development  and  Execution  Process  (FEDEP)  as  standardized  in  IEEE  1516.3  is  an  example  for  applied 
engineering  principles. 

3 

Additional  information  on  this  topic  can  be  found  in  paper  06F-SIW-056  by  Haynes  J,  Maloor  P,  Lyell  M,  and  Zbylut  “A 
Narrative  Approach  to  Simulation-based  Training”,  Proceedings  of  the  Fall  2006  Simulation  Interoperability  Workshop, 
which  was  awarded  by  the  Conference  Committee. 


RTO-MP-MSG-045 


T  - 11 


Technical  Evaluation  Report 


ORGANIZATION 


Simulation 

Model 

r 


Analysis 

T°o1 

/ mx 

1 

!  Trainer  j 

aw# 

-J— r- 

“DIS 

Logging  & 

Rpnla\/ 

2D  Plan  View  / 

to 

Trainer 


3D  Stealth 


Computer  Generated  Simulation 
Forces  Management 


C2IS 

Terrain  Database 
Visual  Database 
Sensor  Database 


Figure  4:  JADE  Architecture  and  Integrated  Systems 


MP-MSG-045-19:  The  fourth  paper  continued  in  the  spirit  of  the  case  study  presentations  of  last  years 
M&S  Symposium  that  clearly  should  the  efficiency  of  M&S  for  training  and  mission  rehearsal  (in 
particular  RTO  MP-MSG-035-008).  It  focuses  on  high-fidelity  multi-player  simulation  for  enhancing 
team,  and  inter-team  skills  to  overcome  constraints  on  range  training  and  complement  live-fly  training  in 
preparation  for  high-cost  training  events.  The  paper  shows  how  in  a  SWE/US  collaboration  Distributed 
Mission  Operations  (DMO)  ideas  and  Mission  Training  via  Distributed  Simulation  (MTDS)  concepts 
were  used  to  integrate  multiple  training  centres  over  long-distance.  The  objective  was  to  evaluate  to  what 
degree  DMO/MTDS  can  complement  Warfighter  training  in  the  SWE  Air  Force  and  the  US  Air  Force. 
The  idea  was  not  to  reduce  flight  hours  but  to  make  the  available  flight  hours  more  efficient  for  education. 
The  Swedish  Defence  Research  Agency  Air  Combat  Simulation  Centre  (FLSC)  in  Kista,  Stockholm,  and 
the  US  Air  Force  Research  Laboratory,  Warfighter  Readiness  Research  Division,  AFRL/HEA  in  Mesa, 
Arizona,  were  utilized  for  the  experimentation.  The  paper  gives  details  on  the  experiments  which  were 
used  to  combine  live  flying  with  DMO  experiments  and  the  use  DMO  experiments  to  improve  tactics. 
These  combinations  allowed  training  in  complex  and  demanding  tactics  that  normally  cannot  be  taught  in 
early  stages  of  the  pilot  education.  Competency-based  training,  defined  as  the  ability  to  compare 
individual  aircrew  performance  to  a  defined  proficiency  level,  to  maintain  acceptable  levels  of 
performance  and  target  areas  requiring  improvement,  proofed  to  be  feasible  and  effective.  Pilots  that 
trained  using  the  principles  explained  in  the  paper  and  then  participated  in  the  live  flying  exercise  Red 
Flag  in  Alaska,  commented  that  “it  felt  like  we  already  have  been  there.” 

MP-MSG-045-20:  The  fifth  paper  presents  Joint  Research  on  Air  Defence  Simulation  (J-ROADS),  which 
is  an  interactive  simulation  environment  on  extended  air  defence  to  support  national  and  international 
armed  forces  with  Analysis  of  System  and  Architecture  Performance,  Exercise  Support  &  Training  and 
Test  Bedding.  J-ROADS  is  based  on  experiences  made  with  SEAROADS  and  was  developed  by  TNO, 
The  Netherlands.  The  objective  of  this  paper  is  similar  to  the  paper  three  of  this  session.  However,  while 
paper  MP-MSG-045-18  focused  on  national  efforts,  this  paper  focuses  on  international  support,  such  as  a 
recent  common  CPX  with  NATO  and  the  Russian  Federation  or  the  USA/NLD/DEU  collaboration  within 
the  Joint  Project  Optic  Windmill  (JPOW).  More  examples  are  given  in  the  paper.  The  target  group  is  the 
Joint  and  Combined  Task  Force  (JCTF)  for  Extended  Air  Defence,  which  is  defined  by  a  heterogeneous 
command  and  control  structure  and  a  multitude  of  engagement  systems.  Paper  and  presentation 
summarize  several  lessons  learned  from  supporting  JPOW.  Of  interest  is  the  Application  of  Global 
Command  and  Control  System  (GCCS)  proved  valuable,  the  NATO  ACCS  functions  demonstrated 
successfully,  and  the  Link- 16  network  proved  capable,  but  provide  major  challenges  to  operate.  The 
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modular  concepts  of  J-ROADS  and  the  integration  of  tactical  data  link  modelling  and  live  systems  is  a 
very  reasonable  candidate  for  a  NATO  solution,  although  the  project  is  still  ongoing  and  some  solutions 
are  not  yet  standardized  or  strictly  follow  NATO  recommended  solutions.  This  paper  is  comparable  to  the 
earlier  presented  paper  MP-MSG-045-18.  Again,  the  main  objective  was  the  efficient  technical  design  for 
the  support  of  the  exercise  cycle,  which  resulted  in  highly  efficient  but  not  necessarily  standard  and 
generally  reusable  solutions. 

MP-MSG-045-21:  The  sixth  paper  has  already  been  referred  to  a  couple  of  times.  It  presents  Modelling  & 
Simulation  Support  to  the  Effects  Based  Approach  to  Operations,  in  particular  from  the  viewpoint  of 
Gamma.  This  paper  reflects  the  observations  of  the  author  and  gives  and  alternative  view  to  the  PMESII 
observations  presented  in  paper  MP-MSG-045-04.  The  paper  gives  several  references  to  earlier 
presentations  describing  architecture,  functionality  and  application  of  the  NATO-developed  tool  for  the 
interested  reader.  The  focus  is  the  transformation  of  experimentation  and  training  with  regard  to  Effect 
Based  Approach  to  Operations  (EBAO),  which  is  the  NATO  term  for  EBO.  The  process  control  model  for 
EBOA  is  of  special  interest  and  is  therefore  shown  here  in  Figure  5. 


System  Centric  EBAO 
Process  Control  Model 


What  is  the 
present  system 
state? 


Start  a  new 
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Action  (COA)  if  not 


Re-plan  if  predicted  system 
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Knowledge  Base 
Development 


Source:  Ufuk  Dirim  NC3A,  Effects  Based  Approach  to  Operation  Conference  April  2006 


Figure  5:  System  Centric  EBAO  Process  Control  Model 

Another  valuable  contribution  of  interest  is  the  distinction  between  the  different  phases  of  EBAO:  It  starts 
with  Effect-based  Planning,  goes  into  the  Effect-based  Execution  and  leads  to  the  Effect-based 
Assessment.  The  role  of  M&S  in  all  three  phases  is  significant,  including  but  not  limited  to  what-if- 
analysis.  Furthermore,  the  agent  model  as  used  in  Gamma  may  serve  as  a  mid-term  alternative  to  a  user- 
driven  incident  list.  As  Gamma  is  used  in  exercises  in  support  of  NATO  staff,  it  may  be  a  good  candidate 
for  decision  support.  Finally,  the  validation  of  Gamma  is  still  very  challenging  and  currently  limited  to 
face-value  evaluation  by  subject  matter  experts.  Although  it  is  planned  to  validate  Gamma  more  formerly, 
this  remains  a  critical  point.  What  makes  Gamma  nearly  unique  is  that  a  conceptual  model  is  captured  not 
only  in  UML  (of  particular  interest  for  the  Verification  part),  but  the  underlying  constraints  and 
assumptions  are  also  captured  in  form  of  the  meta-model  used  for  Zeta.  Zeta  is  a  multi-criteria  decision 
support  tool  embedded  within  Gamma,  linking  effects,  actions  and  resources  to  analyze  predicted 
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outcomes  of  a  selected  civil-military  course  of  action  at  the  operational  level.  In  addition  it  provides 
feedback  on  the  status  of  the  system  under  study  at  the  aggregated  and  actor/element  level  and  the  degree 
of  achievement  of  the  operational  level  military  end- state  over  time. 

MP-MSG-045-22:  The  final  paper  presents  an  overview  on  the  Bulgarian  findings  on  Computer  Assisted 
Exercise  Environment  for  Terrorist  Attack  Consequence  Management,  conducted  as  part  of  a  NATO 
project  in  which  five  other  nations  participate.  The  project  was  supported  by  different  funding  sources, 
which  fits  well  into  several  other  proposed  ideas  on  doing  research  (see  reports  generated  by  several 
participating  groups  connected  with  MSG  interests,  such  as  SISO).  The  focus  of  the  presentation  was  not 
technical,  but  organisational:  What  are  the  different  aspects  than  can  be  supported  by  M&S  in  a  CAX  on 
these  new  topics.  The  resulting  four  topics  are  the  presentation  of  a  CAX  based  model  for  Concept 
Development  and  Experimentation,  the  use  of  training  as  a  tool  for  change  management,  an  evaluation  on 
required  M&S  capabilities  of  the  CAX  environment  (including  the  use  of  COTS),  and  the  use  of  M&S  in 
planning  and  lessons-learned.  The  paper  included  recommendations  for  organisational  structures  in 
support  of  inter-agency  and  inter-ministry  CAX,  focusing  on  structures,  necessary  cells,  integration  of  real 
non-military  command  and  control,  and  necessary  set  up  process.  In  particular  the  J3-organisations  of 
NATO  may  be  interested  in  further  evaluation  of  the  presented  results.  The  symposium  was  used  to 
connect  the  scientists  of  these  efforts  with  those  who  prepared  paper  MP-MSG-045-07,  which  deals  with 
similar  questions. 


8  SUMMARY  AND  RECOMMENDATIONS 

This  section  shows  the  major  trends  as  observed  in  the  papers  as  well  as  in  the  discussions  following  the 
presentations  and  the  sessions.  Each  paragraph  summarizes  a  main  trend  and  gives  a  recommendation  on 
how  NATO  in  general  and  the  NMSG  in  particular  should  act  regarding  the  results  of  the  technical 
evaluation.  In  summary,  the  symposium  provided  attendees  a  forum  to  advance  the  effectiveness  and  role 
of  M&S  in  the  Alliance.  The  combination  of  M&S  users  and  developers  concentrated  in  this  one  forum 
did  not  only  educate  attendees,  but  also  provided  fresh  ideas  for  the  furtherance  of  NATO  M&S  in 
continuing  to  provide  a  key  building  block  in  support  of  NATO’s  transformation  process  and  national 
capabilities. 

•  National  M&S  infrastructures  have  been  established  in  many  nations  can  contribute  to  NATO 
capabilities.  Several  papers  presented  national,  bi-national  and  multinational  efforts  on  setting  up 
such  infrastructure.  All  presenting  nations  made  sure  that  M&S  applications  in  NATO  and  the 
nations  have  reached  a  reliable  level  of  maturity. 

•  The  best  known  application  area  of  M&S  remains  training,  but  the  need  for  support  to  operations 
was  clearly  established  by  military  users.  Several  papers  present  first  applications  into  that 
domain,  such  as  decision  support. 

•  Open  Standards  and  Open  Architectures  have  been  identified  by  several  contributions  as  essential 
for  NATO.  This  was  also  stressed  in  the  discussions.  It  was  also  identified  that  certification  and 
V&V  are  essential.  Several  papers  give  recommendations  in  this  direction. 

•  Web-based  support  was  also  identified  in  several  papers:  repositories,  knowledge-management, 
collaboration  tools,  etc.  are  more  and  more  web-based  and  used  in  a  distributed  style,  including 
workshops  and  symposia.  However,  as  these  are  emerging  standards,  the  guiding  role  of 
coordinating  institutions  -  such  as  MSCO  -  is  essential  to  avoid  non  interoperable  solutions  in 
this  domain. 

•  While  the  current  work  focuses  on  technical  solutions,  process  alignment  is  as  important  and  must 
be  supported.  Education  and  coordination  are  necessary. 

•  In  several  papers,  the  use  of  DoDAF  and  related  frameworks  (MoDAF,  Zachman)  for 
documentation  is  proposed.  However,  the  artefacts  produced  by  such  frameworks  leave  room  for 
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interpretation  and  have  the  danger  of  ambiguity.  The  use  of  UML  has  the  potential  to  make  such 
artefacts  unambiguous  for  NATO  efforts. 

•  Several  papers  focused  on  scalable  solutions  based  on  high-resolution  information  exchange 
paradigm:  JMRM,  MNE4  and  scalable  fidelity  are  examples  for  this  trend.  The  work  is  driven  by 
the  objective  to  use  configurable  and  scalable  solutions  to  address  various  questions  instead  of 
setting  up  individual  federations  to  address  each  question  individually.  Whether  the  current 
approach  to  use  high-resolution  models  for  information  exchange  is  the  best  solution  to  insure 
interoperability  or  if  alternative  approaches  are  needed  remains  to  be  evaluated. 

•  JMRM  is  not  the  final  solution  for  the  flagship  of  MSG,  but  it  is  a  good  step  in  the  right  direction. 
If  done  correctly,  it  can  be  a  first  step  to  the  Pathfinder  vision  of  national  systems  providing  the 
national  capabilities  to  train  users:  the  JMRM  federation  can  be  enriched  by  new  federates. 
However,  a  well  documented  FOM  -  or  even  better  a  standardized  FOM  -  and  the  rigorous 
application  of  open  standards  (such  as  Standards  for  Scenario  Development,  such  as  MSDL, 
standards  for  IE  with  C2,  such  as  C-BML)  is  a  necessary  constraint.  Also,  the  toolset  around 
JMRM  must  support  national  solutions  and  not  exclusively  the  first  core  federates  of  JMRM. 

•  Simulation  Based  Instruction  becomes  increasingly  important.  However,  the  composability 
constraints  for  instruction  modules  may  differ  significantly  from  the  technical  constraints  of  the 
underlying  federation,  so  that  more  research  is  needed.  Nonetheless  it  was  observed  that  M&S 
and  ADL  are  growing  closer  together. 

•  The  often  mentioned  collaboration  between  MSG  and  SISO  and  other  organisations  is  now  a 
reality.  Several  papers  presented  reports  on  mutual  groups  and  the  reuse  of  results  on  both  sides. 

Figure  6  summarizes  the  resulting  recommendations  of  the  technical  evaluator  on  domains  that  need  to  be 
aligned  in  order  to  make  the  Pathfinder  vision  a  reality.4  The  elements  of  the  Pathfinder  house  slide  are 
defined  as  follows: 

•  The  founding  principle  of  Pathfinder  are  Common  Standards  &  Tools,  including  but  not  limited 
to  the  High  Level  Architecture,  command  reference  data  model  -  such  as  the 
C2IEDM/JC3IEDM,-  emerging  web  service.  Based  on  this  principle  the  two  pillars  of  Pathfinder 
are  procedural  and  technical  requirements. 

•  The  procedural  pillar  rests  on  Education.  Just  mandating  and  recommending  technical  standards 
or  management  procedures  is  not  sufficient.  Education  is  a  key  enabler. 

•  The  next  components  are  Aligned  Processes  and  Aligned  Management.  Using  the  same 
elements  only  leads  to  interoperability  and  reusability  if  they  are  used  in  the  same  way.5 

•  The  technical  pillar  rests  on  common  Documentation/Architecture  principles.  Several  papers 
emphasized  this  view  and  it  looks  like  DoDAF  becomes  a  strong  candidate. 

•  The  HLA  Runtime  Infrastructure  -  and  increasingly  emerging  web-based  solutions  -  become  the 
Common  Infrastructure  which  is  used  for  M&S  federations.  This  perspective  may  have  to  be 
broadened  to  be  applicable  to  Command  and  Control  Systems  as  well. 

•  The  use  of  Common  Data,  in  particular  common  data  definitions  and  understanding  including 
metadata  capturing  this  and  enabling  the  mapping  of  data  representation  onto  each  other,  is  part  of 
this  and  a  essential  part  of  effective  net  enabled  components. 


4  A  similar  “house  slide”  was  successfully  used  by  the  US  Army  initiative  Simulation-to-C4I-Systems  Interoperability  (SIMCI) 
to  identify  the  role  of  participating  projects  and  even  for  some  funding  decisions  (see  MP-MSG-022-01). 

5  Paper  MP-MSG-045-16  shows  an  example  how  to  support  this  with  the  Pathfinder  Integration  Environment  Knowledge  Web 
Portal,  in  which  the  M&S  Repository  section  is  used  to  capture  technical  requirements,  and  the  M&S  Knowledge  section  is 
used  to  capture  the  procedural  requirements. 
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•  This  will  lead  to  the  establishment  of  Reusable  Components.  As  highlighted  before,  NATO  now 
sees  a  lot  of  very  mature  national  contributions.  However,  it  appears  many  of  the  common 
problems  are  solved  by  every  nation  individually.  These  resources  can  be  used  more  effectively 
instead  of  “reinventing  the  wheel”  with  national  resources. 


NATO  MSG  Conference  “Transforming  Training  and  Experimentation  through  M&S”,  Slide  3 

Rome,  Italy,  5./6.  October  2006 


Figure  6:  Pathfinder  “House  Slide” 


These  ideas  are  applicable  to  support  the  management  and  evaluation  of  various  technical  activities, 
national  contributions  and  commercial  contributions  to  Pathfinder,  at  least  at  the  high  level,  such  as  within 
MSCO  and  MSG.  An  Integrated  Overarching  Project  Team  could  be  established  to  make  sure  that  the 
Pathfinder  vision  can  become  a  reality  based  on  the  common  practice  within  NATO. 

In  summary,  the  symposium  gave  an  excellent  overview  on  how  much  the  NATO  M&S  Community 
evolved  in  the  recent  years.  All  presenting  nations  and  organisations  are  clearly  in  a  stage  of  mature  M&S 
application.  The  main  challenge  now  may  be  to  effectively  and  efficiently  enable  the  sharing  of  solutions 
based  on  open  standards  and  open  architectures.  While  the  presentations  of  this  symposium  can  be 
characterized  that  all  presenters  showed  mature  M&S  applications  developed  by  the  presenting  nations, 
the  hope  of  the  author  of  this  technical  evaluation  is  that  future  workshops  will  present  national 
contributions  to  a  NATO  wide  framework  representing  a  common  NATO  solution. 


T  - 16 


RTO-MP-MSG-045 


